{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# 损失函数\n",
    "\n",
    "[![](https://gitee.com/mindspore/docs/raw/master/resource/_static/logo_source.png)](https://gitee.com/mindspore/docs/blob/master/tutorials/source_zh_cn/middleclass/custom/loss.ipynb)\n",
    "\n",
    "损失函数用于衡量预测值与真实值差异的程度。深度学习中，模型训练就是通过不停地迭代来缩小损失函数值的过程。因此在模型训练过程中损失函数的选择非常重要，定义一个好的损失函数，可以有效提高模型的性能。\n",
    "\n",
    "MindSpore提供了许多通用损失函数供用户选择，但这些通用损失函数并不适用于所有场景，有时需要用户自定义所需的损失函数。因此，本文介绍损失函数的自定义构建方法。\n",
    "\n",
    "## 自定义损失函数\n",
    "\n",
    "Cell是MindSpore的基本网络单元，可以用于构建网络，损失函数也需要通过Cell来定义。使用Cell定义损失函数的方法与定义一个普通的网络相同，差别在于，其执行逻辑用于计算前向网络输出与真实值之间的误差。\n",
    "\n",
    "以MindSpore提供的损失函数`L1Loss`为例，损失函数的定义方法如下：\n",
    "\n",
    "```python\n",
    "import mindspore.nn as nn\n",
    "import mindspore.ops as ops\n",
    "\n",
    "class L1Loss(nn.Cell):\n",
    "    def __init__(self):\n",
    "        super(L1Loss, self).__init__()\n",
    "        self.abs = ops.Abs()\n",
    "        self.reduce_mean = ops.ReduceMean()\n",
    "\n",
    "    def construct(self, predict, target):\n",
    "        x = self.abs(predict - target)\n",
    "        return self.reduce_mean(x)\n",
    "```\n",
    "\n",
    "在`__init__`方法中实例化所需的算子，并在`construct`中调用这些算子。这样，一个用于计算L1Loss的损失函数就定义好了。\n",
    "\n",
    "代码中使用`nn.Cell`作为L1Loss的基类，最后在`construct`中调用基类提供的`predict, target`方法。`reduction`的合法参数有`mean`、`sum`和`none`，分别表示求均值、求和与输出原值。\n",
    "\n",
    "给定一组预测值`predict`和真实值`target`，调用损失函数，就可以得到这组预测值和真实值之间的差异，如下所示：\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "from mindspore import Tensor\n",
    "\n",
    "loss = L1Loss()\n",
    "input_data = Tensor(np.array([0.1, 0.2, 0.3]).astype(np.float32))\n",
    "target_data = Tensor(np.array([0.1, 0.2, 0.2]).astype(np.float32))\n",
    "\n",
    "output = loss(input_data, target_data)\n",
    "print(output)\n",
    "```\n",
    "\n",
    "以`Ascend`后端为例，输出结果如下：\n",
    "\n",
    "```text\n",
    "0.03333334\n",
    "```\n",
    "\n",
    "在定义损失函数时还可以继承损失函数的基类`Loss`。`Loss`提供了`get_loss`方法，用于对损失值求和或求均值，输出一个标量。\n",
    "\n",
    "L1Loss使用`Loss`作为基类的定义如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import mindspore.ops as ops\n",
    "from mindspore.nn.loss.loss import _Loss\n",
    "\n",
    "\n",
    "class L1Loss(_Loss):\n",
    "    def __init__(self, reduction=\"mean\"):\n",
    "        super(L1Loss, self).__init__(reduction)\n",
    "        self.abs = ops.Abs()\n",
    "\n",
    "    def construct(self, base, target):\n",
    "        x = self.abs(base - target)\n",
    "        return self.get_loss(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "首先，使用`_Loss`作为L1Loss的基类，然后给`__init__`增加一个参数`reduction`，并通过`super`传给基类，最后在`construct`中调用基类提供的`get_loss`方法。`reduction`的合法参数有`mean`、`sum`和`none`，分别表示求均值、求和与输出原值。\n",
    "\n",
    "\n",
    "## 损失函数与模型训练\n",
    "\n",
    "接下来使用定义好的L1Loss进行模型训练。以简单的线性拟场景作为样例。\n",
    "\n",
    "> 线性拟合详细介绍可参考教程[实现简单线性函数拟合](https://www.mindspore.cn/tutorial/training/zh-CN/master/quick_start/linear_regression.html)。\n",
    "\n",
    "### 定义数据集与网络\n",
    "\n",
    "定义训练数据集生成函数，并增强为MindSpore可训练的数据类型。\n",
    "\n",
    "- `get_data`：数据生成函数。\n",
    "- `create_dataset`：将numpy数据转换为MindSpore可训练的函数，并构造数据集的batch增强方法。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from mindspore import dataset as ds\n",
    "\n",
    "# 生成随机数\n",
    "def get_data(num, w=2.0, b=3.0):\n",
    "    for _ in range(num):\n",
    "        x = np.random.uniform(-10.0, 10.0)\n",
    "        noise = np.random.normal(0, 1)\n",
    "        y = x * w + b + noise\n",
    "        yield np.array([x]).astype(np.float32), np.array([y]).astype(np.float32)\n",
    "\n",
    "def create_dataset(num_data, batch_size=16):\n",
    "    dataset = ds.GeneratorDataset(list(get_data(num_data)), column_names=['data', 'label'])\n",
    "    dataset = dataset.batch(batch_size)\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "定义网络，`nn.Dense`将数据集定义为所有的函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from mindspore.common.initializer import Normal\n",
    "import mindspore.nn as nn\n",
    "\n",
    "class LinearNet(nn.Cell):\n",
    "    def __init__(self):\n",
    "        super(LinearNet, self).__init__()\n",
    "        self.fc = nn.Dense(1, 1, Normal(0.02), Normal(0.02))\n",
    "\n",
    "    def construct(self, x):\n",
    "        return self.fc(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 使用Model进行模型训练\n",
    "\n",
    "`Model`是MindSpore提供的用于模型训练、评估和推理的高阶API。创建数据集并定义一个`Model`就可以使用`train`接口进行模型训练。接下来我们使用`Model`进行模型训练，并采用之前定义好的`L1Loss`作为此次训练的损失函数。\n",
    "\n",
    "使用之前定义的`LinearNet`和`L1Loss`作为前向网络和损失函数，并选择MindSpore提供的`Momemtum`作为优化器。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "net = LinearNet()\n",
    "loss = L1Loss()\n",
    "opt = nn.Momentum(net.trainable_params(), learning_rate=0.005, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "定义`Model`时需要指定前向网络、损失函数和优化器，`Model`内部会将它们关联起来，组成一张训练网。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [],
   "source": [
    "from mindspore import Model\n",
    "\n",
    "model = Model(net, loss, opt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "创建数据集，并调用`train`接口进行模型训练。\n",
    "\n",
    "参数解释：\n",
    "\n",
    "- `epoch`：训练数据集的迭代次数。\n",
    "- `train_dataset`：训练数据集。\n",
    "- `callbacks`：`model.train`的回调函数参数，也可不使用回调函数功能。\n",
    "- `LossMonitor`：损失函数值监视器，用于打印训练过程中的模型损失值。\n",
    "- `dataset_sink_mode`（bool）：数据下沉模式，可加快训练。Ascend和GPU平台支持开启该功能（True）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "jupyter": {
     "outputs_hidden": false
    },
    "pycharm": {
     "name": "#%%\n"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 1 step: 1, loss is 8.042041\n",
      "epoch: 1 step: 2, loss is 10.255897\n",
      "epoch: 1 step: 3, loss is 11.757828\n",
      "epoch: 1 step: 4, loss is 10.169484\n",
      "epoch: 1 step: 5, loss is 8.056287\n",
      "epoch: 1 step: 6, loss is 7.80862\n",
      "epoch: 1 step: 7, loss is 8.432369\n",
      "epoch: 1 step: 8, loss is 8.16351\n",
      "epoch: 1 step: 9, loss is 6.4131327\n",
      "epoch: 1 step: 10, loss is 4.8829217\n"
     ]
    }
   ],
   "source": [
    "from mindspore.train.callback import LossMonitor\n",
    "\n",
    "ds_train = create_dataset(num_data=160)\n",
    "model.train(epoch=1, train_dataset=ds_train, callbacks=[LossMonitor()], dataset_sink_mode=False)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "MindSpore1.2",
   "language": "python",
   "name": "mindspore1.2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
